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and  Production  Acceptance  Tests:  Exponential  Distribution"  [2] 
covers  the  requirements  for  reliability  qualification  tests 
(pre-production)  and  reliability  acceptance  tests  (production)  for 
equipment  that  experiences  a  distribution  of  times-to-failure  that 
is  exponential.  These  requirements  include:  test  conditions, 
procedures,  and  various  fixed- length  and  sequential  test  plans 
with  respective  accept /reject  criteria.  This  paper  is  concerned  only 
with  the  statistical  test  plans  and  the  selection  and  use  of  these 
plans..  The  Standard  contains  both  fixed-length  test  plans  (Plans 
IXC  through  XVIIC  and  XIXC  through  XXIC)  and  probability-ratio 
sequential  tests  (Plans  IC  through  VII1C  and  XVIIIC).  Each  fixed- 
length  test  plan  is  characterized  by  its  discrimination  ratio  (d) , 
its  total  test  time  (T) ,  and  its  maximum  allowable  number  of  failures 
to  accept  (k).  If  a  fixed-length  test  plan  is  selected,  the  total 
test  duration  is  essentially  set  in  advance.  The  only  way  in  which 
one  of  these  plans  can  terminate  early  is  by  rejection.  For  example, 
Test  Plan  XVIIC  terminates  with  a  reject  decision  at  the  third 
failure  if  this  failure  occurs  before  4.3  units  of  totsl  test  time 

1/  Work  supported  in  part  by  Contract  N00014-79-C-0751  with  the  Office 
of  Naval  Research. 


— -*J  nv  rui  i-  A-i  -  Twiiiiur  . 


<•  4  •*£**&«  Lt 


havu  transpired.  An  accept  decision  can  only  be  made  when  4.3  units 
of  total  test  time  have  accrued.  Even  if  the  second  failure  occurs 
very  early,  an  early  reject  decision  cannot  be  made;  nor  can  an  early- 
accept  decision  be  made  if  no  failures  have  occurred,  say,  by  time  4.0. 
In  both  of  these  situations,  an  early  decision  would  lack  statistical 
validity  in  failing  to  guarantee  the  operating  characteristic  of  the 
selected  plan.  Moreover,  an  early  reject  decision  by  the  consumer 
would  probably  violate  contractual  agreements  with  the  producer. 
However,  an  early-accept  decision  by  the  consumer  would  not  be  subject 
to  such  an  objection.  Such  a  decision  might  seem  very  desirable  to 
the  consumer  (government)  if  testing  costs  were  substantial  or  if 
schedule  deadlines  were  near.  This  paper  presents  modifications  to 
the  fixed-length  test  plans  of  MIL-STD-781C  which  allow  early-accept 
decisions  to  be  made  without  sacrificing^  statistical  validity.  The 
proposed  plans  differ  from  the  probability  ratio  sequential  tests  in 
the  Standard  in  that  rejection  is  permitted  only  after  a  fixed  number 
of  failures  have  been  observed. 

2.  The  Early-Accept  Criterion 

The  early-accept  criterion  we  w511  consider  is  as  follows. 
Consider  a  test  plan  $P.  with  discrimination  radio  d.  total  test 
time  T^,  maximum  allowable  number  of  failures  to  accept  k  (k  _>  1), 
and  consumer's  risk  6.  Consider  alternative  test  plans 
&q*  •••*  <^-1  with  the  same  discrimination  ratio,  maximum 

allowable  number  of  failures  to  accept  j  (0  <_  j  <  k) ,  and  total  test 


tlM»  T j  -  f  *  x(l-8t2j+2)*  X(l-e,2j+2)  U  th*  (1  “ 

percentile  of  a  chi-squared  distribution  with  2j+2  degress  of 

freedom.—1^  Tha  producer's  risks  for  tsst  plans  £  J  £  k)  ara 

in  dacraaaing  ordar  of  J,  tha  tast  tlaaa  ara  in  lncraasing  ordar  of 
j,  av.d  tha  consumer's  risks  are  constant  in  j  (each  Is  (J). 

Tha  aarly-accapt  criterion  is  as  follows:  accept  at  time  Tj  if 
at  aost  j  failures  have  occurred  up  to  that  tins.  This  alters  the 
original  test  plan  by  allowing  early-accapt  decisions  to  be 

made  at  k  time  points  prior  to  the  tot~l  test  time  T^.  As  a 
result  the  producer's  risk  for  test  plan  ^  is  altered.  Also, 
even  though  each  test  plan  has  consumer's  risk  0, 

and  even  though  the  alternative  test  plans  were 

only  involved  with  accept  decisions,  the  consumer's  risk  of  the 
resulting  test  is  not  maintained  at  6,  and  indeed,  may  be 
significantly  greater  than  g.  It  is  true  that  if  an  early-accept 
decision  is  made  at  time  T^,  then  test  plan  had  it  been 

selected  prior  to  the  start  of  testing,  would  have  reached  the  same 
conclusion.  But,  by  allowing  the  test  results  to  effectively  dictate 
which  test  plan  is  used,  the  probability  calculations  involved  in 
determining  the  consumer's  risk  are  modified  by  the  conditional 
probabilities  which  must  consequently  be  incorporated  into  them. 


— ^This  choice  is  somewhat  arbitrary,  but  is  motivated  by  the  use  of 
this  rule  to  guarantee  a  given  consumer's  risk  for  a  fixed-length 
test  plan. 
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Th«  producer's  end  consumer's  risks  for  the  modified 

tsst  plans  sra  computed  es  follows.  Let  P^(X)  denote  the 

probability  of  accepting  when  the  true  mean  time  between  failures 


(MTBF)  is  1/X. 


■e 

P  (X)  ■  Y  Pr{eccept  at  time  T.) 
K  J-0  J 


Let  A>.v  “  Pr{eccept  at  time  T^}. 


Theorem  1.  Suppose  the  true  MTBF  is  1/X.  Then 


r,)J.»(-XI4)  j-l  [X(T,-T.)]J'*.xp(-X(T,-T 


Proof.  If  an  accept  decision  is  made  at  time  T^,  then  exactly  *. 
failures  must  have  occurred  up  to  that  time  (since  if  fewer  than 
A  failures  had  occurred,  an  accept  decision  v>uld  have  been  made 


earlier) .  Thus 


Pr{ exactly  j  failures  in  [O.Tj]}  -  Pr  accept  at  time  T£  and 

(j-A)  failures  in  (T^.T^]} 

where  represents  a  union  of  disjoint  events. 


(XT.)^exp(-XT.)  j-l 


[la-i/V-Kyy;  »  n 


Hi*  comumi'i  risk  for  tha  early-accept  taat  plan  la  PA(1)  and 
tha  producer's  rlak  la  1  -  P^(l/d). 

3.  Barly-Accept  Taat  Plana 

It  haa  baan  propoaad  that  tha  early-accept  critarlon  bn  uaad 
with  tha  axlatlng  parameters  of  tha  fixed- length  taat  plana  of 
MIL-STD-781C.  Tha  affaet  of  Incorporating  tha  early-accept  critarlon 
Into  thasa  fixed-length  taat  plana  (without  furthar  modification)  la 
ahown  in  Tabla  1.  In  all  plans  axcapt  Plan  XXIC  tho  consumer's  risk 
Is  Increased  and  the  producer's  risk  is  decreased.  (Test  Plan  XXIC 
is  unchanged  since  It  only  accepts  whan  there  era  no  failures.)  The 
changes  are  substantial;  often  the  consumer's  risk  is  more  than 
doubled  and  the  producer's  risk  halved.  By  altering  the  test  time 
and  the  maximum  number  of  failures  to  accept,  it  is  possible  to 
correct  for  the  effect  of  the  early-accept  modification  and  closely 
match  the  operating  characteristics  (at  two  points)  of  the  standard 
fixed-length  test  plans.  The  corrections  for  each  of  tha  MIL-STD-781C 
fixed- length  test  plans  are  given  in  Table  2.  Accept  times  for  these 
early-accept  test  plans  are  listed  in  Table  3. 

The  corrections  were  computed  by  defining  functions  fQ(T,k) 

as  the  producer's  risk  for  an  early-accept  test  plan  with  parameters 

T  and  k,  and  fg(T,k)  as  the  consumer's  risk.  As  T  increases 

f  increases  and  fD  decreases,  and  as  k  increases  f  decreases 
o  p  a 

and  fg  increases.  Because  of  the  integer  restriction  on  k,  it  is 
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not  always  possible  to  design  a  test  plan  to  schisvs  specified 
value*  of  a,  B  exactly.  However,  an  algorithm  which  will 
determine  an  approximate  solution  can  be  constructed.  The  algorithm 
from  which  Table  2  is  derived  first  fixes  k  and  uses  a  quasi-Newton 
method  to  determine  a  vclue  of  T  which  will  achieve  the  desired 
a- value.  The  process  in  then  repeated,  varying  k  in  accordance 
with  a  bisection  search,  to  determine  a  k-value  for  which  6  is 
also  close  to  the  desired  level.  Some  additional  checks  to  reduce 
the  calculations  a**  also  Incorporated.  It  should  be  noted  that 
the  test  plans  of  Table  2  are  designed  to  have  a  and  B  levels 
close  to  the  nominal  values  of  the  standard  test  plana,  not  the 
actual  values.  (See  Tables  II  and  C-l  in  t2]). 


4.  Performance  of  the  Early-Accept  Test  Plans 

Table  2  shows  that  the  maximum  test  times  for  the  early-accept 
test  plans  are  substantially  increased  from  the  standard  teat  times. 
However,  the  expected  test  times  for  the  early-accept  plans  are 
much  smaller  than  the  maximum  times,  and  compare  quite  favorably 
to  the  (fixed)  test  times  for  the  standard  planar^  Graphs  of  expectsd 
test  duration  versus  true  MTBF  for  the  early-accept  test  plans  appear 
in  Figures  1-12.  If  estimation  of  the  truo  MTBF  is  not  necessary,  a 
regular  fixed-length  test  can  be  terminated  "early"  when  rejection  is 
the  appropriate  action.  Therefore,  each  figure  also  graphs  the  expected 
test  duration  versus  the  true  MTBF  for  this  procedure.  It  is  not 

1/  The  expected  test  times  for  Early-Accept  Plans  IXC  and  XC  exceed  those 
for  the  corresponding  standard  plans  for  a  considerable  range  of  the 
true  MTBF.  The  reason  for  this  is  that  these  two  early-accept  plans 
have  producer's  and  consumer'!*  risks  substantially  closer  to  the  nominal 
values  than  do  the  standard  plans. 


surprising  that  the  early-aceept  teat  plans  gana rally  hava  smeller 
expected  tast  durations. 

Tha  expected  tast  durations  ara  coaputad  as  follows.  Lat  t 
ba  tha  (random)  tast  duration. 
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where  I  danotas  tha  indicator  function  of  the  ovont  B,  l.e.,  Ig 
B 

equals  1  if  the  event  occurs,  0  otherwise.  To  compute  the  ".arms  in 

tha  second  sumatlou  in  (1),  note  that  for  the  plan  reject  in 

(T  ,T  ],  r  failures  (J  <  r<k)  must  occur  prior  to  T^,  and 
J  *  J 

k  +  1  *  r  failures  must  occur  in  ®»u** 
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(T^  ♦  s)f(s)  da. 


inhere  f  (*)  ia  a  gamma  density  with  parameters  X,  k  +  1  -  r,  and 
Q(j»t)  “  Pr  {do  not  accept  or  reject  at  or  before  time  T^,  and  r 
failures  in  [0,T,j]},  (J  <  r). 


(2) 


Theorem  2.  For  J  <  r  £  k 


(XT4)rexp(-XT4)  j;l  tXCT^-T^)]1  ^expC-XOj-T^)) 

Q(3.r)  -  — “-n — L  -  Jo  *<‘) - 1 - uhTH — J —  • 


Proof.  Pot  J  <  v  <  k, 
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All  that  remains  la  to  compute  cha  Intagral  In  (2).  This 
Integral  can  be  axpraaaad  In  teraa  of  tha  Incomplete  ganma 
dlatrlbutlon  and  evaluated  by  utandard  computer  vubroutines  (1). 
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Let  I(x,p)  be  the  Incomplete  game  dlatrlbutlon,  that  la 


Kk 


> 

J  ~TW~ 


dt. 


8 


f 


Wl 


»‘W»> 


dj.1+.>f(.)d.  -  tj.!  •  Kxttj-tj.,).  *-«+i>  DfTfear1- 

0 


/»  J  J  J 
l5I  T(k-r+2) 


k-r+1 


TJml  •  KAITj-^3,  k-r+1)  +  KXITj-Tj^J,  k-r+2) 


5.  Conclusions 

Froi  an  operating  charaetariatlc  curva  point  of  view,  it  doaa  not 
make  ouch  difference  whathar  a  fixad-langth  taat  plan,  a  probability- 
ratio  aaquantial  taat  plan  (truncatad)  or  an  aarly-accapt  plan  la  chosen, 
provided  each  is  daaignad  to  hava  tha  aane  oparatlng  charactarlatlcs. 
Generally,  tha  ordarlng  of  tha  axpacted  taat  duration  la  amalleat  for 
tha  probability  ratio  aaquantial  taat  plan,  followed  by  tha  aarly-accapt 
plan,  and  largaat  for  tha  fixad-langth  plan.  Tha  advantage  of  tha  aarly- 
accapt  plan,  over  tha  probability  ratio  aaquantial  taat  plan,  ia  purely 
paychologlcal.  Tha  producer  never  haa  a  lot  rejected  early,  and  early 
decisions  occur  only  with  the  oaalrable  outcome  (from  the  producer's 
point  of  view)  of  acceptance.  Such  an  advantage  cannot  be  discounted. 
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TABLE  1  -  Changes  In  Producer's  and  Consumer's  Risks 
Resulting  from  Incorporating  Early-Accept 
Criterion  into  MIL-STD-781C  Tost  Plans 


Test  Discrimination 
Plan  Ratio 


IXC 

1.5 

SB 

1.5 

XIC 

1.5 

XIIC 

2.0 

XIIIC 

2.0 

XIVC 

2.0 

XVC 

3.0 

XVIC 

3.0 

XVI IC 

3.0 

(High  Risk 
Plans) 

XIXC 

1.5 

XXC 

2.0 

XXIC 

3.0 

* 

Without  Early-Accept  Option 
Producer's  Consumer's 


Risk  (%) 
12.0 
10.9 

17.8 
9.6 
9.8 

19.9 
9.4 

10.9 
17.5 


Risk  (X) 
9.9 
21.4 
22.1 
10.6 
20.9 
21.0 
9.9 
21.3 
19.7 


28.8  31.3 
28.8  28.5 
30.7  33.3 


With  Early-Accept  Option* 
Producer's  Consumer's 


Risk  (X) 

4.9 
3.5 
6.8 

4.7 
*  4.4 

11.3 

5.9 

6.8 

12.5 


Risk  (%) 

38.1 

58.8 

56.4 

31.8 

48.4 

42.8 

23.1 

38.4 
32.6 


14.0  59.5 
19.4  44.6 
30.7  33.3 


•k 

Taken  from  Tables  II  and  III  of  MIL-STD-781C  and  is  for  the  test  plan 
without  early-accept  modification. 

*True  risk  when  the  early-accept  criterion  is  incorporated. 


11 


TABLE  3  -  Accept  Times  of  Barly-Accept  Test  Plans 


Test  Plan  Accept  Times* 


IXC 

Tq  -  4.2 

Tr 

■  6.1 

T2  -  7.9 

T3  -  9.4 

t4  -  11.0 

T5  -  12.4 

Tfi  -  13.9 

T?  -  15.3 

Tg  -  16.6 

T9  -  18.0 

T10 

-  19.3 

TU 

-  20.7 

T12 

-  22.0 

T13 

-  23.3 

T14 

-  24.5 

T15 

-  25.8 

T16 

-  27.1 

T17 

-  28.3 

T18 

-  29.6 

X19 

-  30.8 

T20 

-  32.1 

T21 

-  33.3 

T22 

-  34.5 

T23 

-  35.8 

T24 

-  37.0 

T25 

-  38.2 

T26 

-  39.4 

T27 

-  40.6 

T28 

-  41.8 

T29 

-  43.0 

T30 

-  44.2 

T31 

-  45.4 

T32 

-  46.6 

T33 

-  47.8 

T34 

«  49.0 

T35 

-  50.1 

T36 

-  51.3 

T37 

-  52.5 

T38 

-  53,7 

T39 

-  54.8 

T40 

-  56.0 

T41 

-  57.2 

T42 

-  58.3 

T43 

-  59.5 

T44 

-  60.7 

T 

X45 

-61.8 

T46 

-  63.0 

T47 

-64.1 

T48 

-  65.3 

T49 

-  66.5 

T50 

-  67.6 

X51 

-  68.8 

T52 

-  69.9 

T53 

-  71.1 

T54 

-  72.2 

xc 

To" 

■  3.2 

Tl- 

-  5.0 

T2 

6.6 

T3 

'  8.1 

T4- 

■  9.5 

T5 

'  10.9 

T6 

■  12.2 

T7 

13.6 

T  ■ 

8 

•  14.9 

T9 

■  16.1 

T10 

-  17.4 

T 

11 

-  18.7 

T12 

-  19.9 

T13 

-  21.2 

T14 

-  22.4 

T15 

-  23.6 

T 

i6 

-  24.8 

f17 

-  26.1 

T18 

-  27.3 

T19 

-  28.4 

Cont 

'd. 

Accept  at 

time 

Tj  if 

j  failures 

have 

occurred  to 

i  that  time. 
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TABLE  3  (Cont’d) 


I,.t  Plan  tecapt  Tla».« 


xc 

T  * 

*20 

29.6 

T  * 

*21 

30.8 

*22  ‘ 

32.0 

T  ■ 

23 

33.2 

T24- 

34.4 

(Cont'd) 

T  m 

T25 

35.6 

T26 

36.7 

T27  " 

37.9 

T  _  aa 

*28 

39.1 

*29  ' 

40.2 

T30  “ 

41.4 

T31 

42.5 

*32  ’ 

43.7 

T  « 

a33 

44.8 

T34- 

46.0 

T35 

47 ,1 

T36  “ 

48.3 

T37  “ 

49.4 

T38  “ 

50.6 

*39  * 

51.7 

XIC 

To“ 

3.0 

Tl“ 

4.8 

t2  -  < 

5.3 

T3 

7.8 

T4“ 

9.2 

T5 

10.5 

T6 

11.9 

T7": 

13.2 

T8 

14.4 

T9“ 

15.7 

T10" 

>  17.0 

T11  "  18,2 

T  * 

i2 

19.5 

T13 

20.7 

T14- 

21.9 

T15  ‘  “-1 

*16  ■  24'3 

T  ■ 

*17 

25.5 

rP  n 

*18 

26.7 

t  ■ 

*3.9 

27.9 

t20  '  29-1 

T21  =  30.3 

*22  ■ 

31.4 

T  * 

*23 

32.6 

XIIC 

To“ 

3.7 

T  ti. 

l 

5.6 

T2  ** 

7.2 

*3  “ 

8.8 

T4  “ 

10.3 

T5 

11.7 

T6“ 

13.1 

T7 

14.4 

T8 

15.8 

T  * 

*9 

17.1 

T10 

=  13.4 

TU 

-  19.7 

*12  *  21'° 

*13  -  22'3 

T14- 

23.5 

T15 

-  24.8 

T16 

-  26.0 

XIIIC 

To' 

2.8 

Tl" 

>  4.6 

T  * 

6.1 

T  * 

3 

7.5 

T4 

8.9 

T5 

■  10.3 

T6 

•  11.6 

T  is 

*7 

12.9 

T  m 

8 

14.1 

T  w 

*9 

15.4 

T10 

-  16.6 

T11 

-  17.9 

T12 

-  19.1 

#r 

Accept  at 

time 

*j  if 

j  failures  have 

occurred  to 

that 

time. 
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TABLE  3  (Coat’d) 


Teat  Plan 

Accept  Tlaes* 

XIVC 

To- 

2.7 

-  4.4 

T2  -  5.9 

T3  -  7.3 

X4 

-  8.7 

Tj  • 

10.0 

T6  -  11.3 

T?  -  12.6 

XVC 

To- 

3.5 

Tx  -  5.4 

T2  -  7.0 

T3  -  8.6 

T4 

-  10J 

T5 

11.4 

T6  -  12.8 

XVIC 

To- 

2.5 

T1  -  4.1 

T2  -  5.6 

T3  -  7.0 

X4 

-  8.3 

XVIIC 

To“ 

2.2 

Tx  -  3.8 

I2  -  5.2 

XIXC 

To- 

2.1 

Tx  -  3,7 

T2  »  5.1 

T3  -  6.4 

X4 

-  7.7 

t5  - 

5,9 

'i!6  «  10.2 

T?  -  11.4 

*8  *  12*6 

xxc 

T  « 

o 

1.8 

Tx  -  3.2 

T2  -  4.5 

XXIC  Tq  *  1.1 


Accept  at  time 


if  J 


failures  have  occurred  to  that  tltaa. 


15 


1.00 '  1  1  1  1  '  Loo*  0  '  '  '  *  *  *  too*  1  1  '  1  ' 
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TEST  PLAN  IXC 


Figure  1 
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TEST  PLAN  XC 


Figure  2 
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Figure  3 
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Standard  Fixed-Length  Teat  Plan 


TRUE  MTV  (XN  MULTIPLES  V  LOWER  TEST  MTSF) 

TEST  PLAN  XIIC 


Figure  4 
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TEST  PLAN  XIIIC 
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THUS  MTV  UN  MULTVUCS  V  LOUD*  TOT  MTV) 


TEST  PLAN  XIVC 


Figure  6 


Figure  7 
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TEST  PLAN  XVIC 


Figure  8 
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tudard  Fixad-Langth  Teat  Plan 
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TEST  PLAN  XVIIC 


Flgura  9 
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TRUE  MTBF  (IN  MULTIPLES  OF  LOUER  TEST  MTBF) 

TEST  PLAN  XIXC 

Figur.  10 
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EXPECTED  TEST  DURATION  (IN  MULTIPLES  OF  LOWER  TEST  MTBFJ 


EXPECTED  TEST  DURATION  (IN  MULTIPLES  OF  LOWER  TEST  MTBF) 


TEST  PLAN  XXIC 

^  Figure  12 

The  Standard  Fixed-Length  Test  Plan  with  Early  Rejection  and  the  Early-Accept 
Teat  Plan  are  Identical  for  this  case. 
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(Plans  IC  through  VIIIC  and  XVIIIC).  Each  fixed-length  test  plan  is 
characterized  by  its  discrimination  ratio  (d) ,  its  total  test  time  (T) ,  ..\A  its 
maximum  allowable  number  of  failures  to  accept  (k).  If  &  fixed-length  test  plan 
is  selected,  the  total  test  duration  is  essentially  set  in  advance.  The  only 
way  in  which  one  of  these  plans  can  terminate  early  is  by  rejection.  For 
example,  Test  Plan  XVIIC  terminates  with  a  reject  decision  at  the  third  failure 
if  this  failure  occurs  before  4.3  units  of  total  test  time  have  transpired. 

An  accept  decision  can  only  be  made  when  4.3  units  of  total  test  time  have 
accrued.  Even  if  the  second  failure  occurs  very  early,  an  early  reject  decision 
cannot  be  made;  nor  can  an  early-accept  decision  be  made  if  no  failures  have 
occurred,  say,  by  time  4.0.  In  both  of  these  situations,  an  early  decision 
would  lack  statistical  validity  in  failing  to  guarantee  the  operating  charac¬ 
teristic  of  the  selected  plan.  Moreover,  an  early  reject  decision  by  the 
consumer  would  probably  violate  contractual  agreements  with  the  producer.  How¬ 
ever,  an  early-accept  decision  by  the  consumer  would  not  be  subject  to  such  an 
objection.  Such  a  decision  might  seem  very  desirable  to  the  consumer 
(government)  if  testing  costs  were  substantial  or  if  schedule  deadlines  were 

i 

;  near.  This  paper  presents  modifications  to  the  fixed-length  test  plans  of 

i 

|  MIL-STD-781C  which  allow  early-accept  decisions  to  be  made  without  sacrificing 
statistical  validity.  The  proposed  plans  differ  from  the  probability  ratio 
sequential  tests  in  the  Standard  in  that  rejection  is  permitted  only  after  a 
fixed  number  of  failures  have  been  observed. 


